Compression encoding technology is used in order to efficiently transmit and store still image data and moving image data. The MPEG-1 to 4 and ITU (International Telecommunication Union) H.261 to H.264 systems are widely used for moving images.
In such encoding systems, encoding processing and decoding processing are carried out after dividing an image to be encoded into a plurality of blocks. In prediction encoding within a picture, a prediction signal is produced by using an adjacent already reproduced image signal (obtained by restoring compressed image data) within the same picture where a target block is included, and thereafter a differential signal obtained by subtracting the prediction signal from a signal of the target block is encoded. In prediction encoding between pictures, referring to an adjacent already reproduced image signal within a different picture from the picture within which the target block is included, motion compensation is carried out, and a prediction signal is produced. The prediction signal is subtracted from the signal of the target block so as to be a differential signal and the differential signal is encoded.
For example, intra-picture prediction encoding of H.264 adopts a method in which a prediction signal is produced by extrapolating, in a predetermined direction, already reproduced pixel values adjacent to a block to be encoded. FIG. 22 is a schematic view to describe an intra-picture prediction method used in ITU H.264. In FIG. 22(A), a target block 1802 is an encoding target block. A pixel group 1801 is an adjacent region composed of pixels A to M located adjacent to the boundary of the target block 1802. The pixel group 1801 is an image signal that has been already reproduced in a past process.
In this case, the pixel group 1801 that is composed of adjacent pixels directly above the target block 1802 is extrapolated downwards to produce a prediction signal. In FIG. 22(B), the already reproduced pixels (I to L) located on the left of a target block 1804 are extrapolated rightward to produce a prediction signal. The specific method of producing a prediction signal is described in Patent Document 1, for example. The difference with the pixel signal of the target block is found for each of the nine prediction signals produced by the method shown in FIG. 22(A) to FIG. 22(I) as described above. The prediction signal having the lowest difference is taken to be the optimal prediction signal. In this way, a prediction signal can be produced by extrapolating pixels. The details are explained in Patent Document 1 listed below.
In typical inter-picture prediction encoding, a prediction signal is produced by a method that searches for a signal resembling the pixel signal of a block to be encoded from already reproduced pictures. A motion vector and a residual signal are encoded. The motion vector indicates the spatial displacement amount between the target block and a region constituted by the searched signal. The residual signal means the difference between the pixel signal of the target block and the prediction signal. The technique of searching for the motion vector for each block in this way is called block matching.
FIG. 5 is a schematic view to describe a block matching process. Here, the procedure for producing a prediction signal is described by taking as an example a target block 402 on a target picture 401. A picture 403 has been already reproduced. A region 404 is a region that is in the same spatial position as the target block 402. In block matching, a search range 405 surrounding the region 404 is set, and then a region 406 that has the lowest sum of absolute differences between the pixel signal of the target range and the pixel signal of the target block 402 is detected from the search range 405. The signal of the region 406 becomes a prediction signal. The displacement amount from the region 404 to the region 406 is detected as a motion vector 407. In H.264, in order to correspond to local feature changes in images, a plurality of prediction types with different block sizes for encoding the motion vector is prepared. The prediction types of H.264 are described in Patent Document 2, for example.
In compression encoding of moving image data, each picture (frame or field) can be encoded in any sequence. Therefore, there are three approaches on encoding sequence in the inter-picture prediction that produces a prediction signal using reproduced pictures. The first approach is a forward prediction that produces a prediction signal using reproduced pictures in the past in the order of reproduction sequence. The second approach is a backward prediction that refers to reproduced pictures in the future in the order of reproduction sequence. The third approach is a bidirectional prediction that carries out both the forward prediction and the backward prediction so as to average the two prediction signals. The types of inter-picture prediction are described in Patent Document 3, for example.
As described above, H.264 provides the prediction types with different block sizes for encoding the motion vector in order to cope with local feature changes in images. As the block size shrinks, the number of motion vectors to be encoded increases, which has led the development of a method of producing motion vectors of four blocks divided from an encoding target block by utilizing already reproduced pixel values adjacent to each block based on one motion vector. This method is described in Patent Document 4.
Additionally, there is a method of producing a prediction signal by utilizing already reproduced pixel values adjacent to an encoding target block without the motion vector. The method can remove noise components included in prediction signals by producing and averaging a plurality of candidate prediction signals, like in the bidirectional prediction. This method is described in Patent Document 5.